Since it still requires a folder structure for orangization

Could you explain this part? I didn't expect that anything would need to move. You could just do a simple download of each dataset from Girder, then point to the data in-place without moving anything.

Also, I expect you'd like to merge this before #487. I'm fine with that. Just to confirm, this will work with arbitrary dataset ids right? They don't have to be siblings?

Could you explain this part? I didn't expect that anything would need to move. You could just do a simple download of each dataset from Girder, then point to the data in-place without moving anything.

besides my massive spelling mistake there (orangization). Bad choice of words for the explanation. I had the assumption that the testing of the ground_truth to see if it is a directory was in there because of some legacy items where the meta.detection might provide a directory instead of a csv so that test needed to remain in there. Really all organize_folder_for_training does right now is check if the ground_truth is a directory, if it is it will copy the first .csv out of it into the training data directory associated with that groundtruth and then deletes the folder. If it is a file it just renames it to ground_truth.csv which is unnecessary but cleaner. I could keep all ground truth at the root level, but if I'm doing the test and possibly moving it why not keep it a bit more organized. I should probably clean up the description and what it is called.

Also, I expect you'd like to merge this before #487. I'm fine with that. Just to confirm, this will work with arbitrary dataset ids right? They don't have to be siblings?

Yeah that was the second part of my testing, I was training across different user's public folders, just required that I manually create the array of dataset ids and call the endpoint because there is no UI for it currently.

Really all organize_folder_for_training does right now is check if the ground_truth is a directory, if it is it will copy the first .csv out of it into the training data directory associated with that groundtruth and then deletes the folder. If it is a file it just renames it to ground_truth.csv which is unnecessary but cleaner.

For the record, the reason this is done is in case the girder item has multiple files. If it does, it's a folder when downloaded. Otherwise it's just a file. Since we still use the csv_detection_file method to ensure a csv file on the item, this is likely always the case.